Skip to content

fix(qa): give the expression ledger a fail-policy for "nothing evaluates this" - #16660

Merged
os-sales merged 1 commit into
mainfrom
claude/issue-15533-failpolicies-unevaluated
Sep 7, 2026
Merged

fix(qa): give the expression ledger a fail-policy for "nothing evaluates this"#16660
os-sales merged 1 commit into
mainfrom
claude/issue-15533-failpolicies-unevaluated

Conversation

@claude

@claude claude Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Fixes #15533

Ruling: option C, director seat, decision batch #73, 2026-09-07, maintainer verbatim 「同意」, recorded at comment 5565954678. FAIL_POLICIES gains an explicit member meaning "nothing evaluates this slot" — unevaluated — and all five state: 'experimental' rows are re-stated with it, cel-advanced-policy included. Option B (spread fail-closed) was refused; option A is what shipped and is what this replaces.

What changed

Two files, both inside the private @objectstack/dogfood workspace:

  • packages/qa/dogfood/test/expression-conformance.ledger.tsFailPolicy gains unevaluated with a docblock defining it; the five rows are re-stated; two notes whose prose defended the borrowed value are re-written.
  • packages/qa/dogfood/test/expression-conformance.test.tsFAIL_POLICIES gains the member, and a new pin refuses the shapes the word could be borrowed in.

The vocabulary's locality was re-derived from the delivered diff rather than inherited: git grep FailPolicy and git grep EXPRESSION_SURFACE return hits in those two files only, and fail-soft-log appears nowhere else in the tree. Nothing published moves. Clause-② stays no.

The ruling's census was taken on a pre-#15802 tree — re-derived here

The ruling scopes the work as "re-read 20 rows" (4 compile-error · 7 fail-closed · 7 fail-soft-log · 2 throw). #15500 closed on 2026-09-05 via PR #15802, whose whole purpose was to give every declaration its own ratchet key — which splits previously-collapsed keys into separate rows. The ruling comment is dated 2026-09-07T06:20:23Z — two days later — so it could not carry that arithmetic. (Dates verified against the API rather than inherited: #15500 closed_at 2026-09-05T10:43:34Z; PR #15802 merged_at 2026-09-05T10:43:33Z, title "fix(qa): give every expression declaration its own ratchet key".)

Re-derived on this branch's merge base by importing the ledger and counting, not by reading the ruling:

failPolicy ruling (pre-#15802) today, before this PR after this PR
compile-error 4 4 0
fail-closed 7 8 7
fail-soft-log 7 9 9
throw 2 2 2
unevaluated 5
total rows 20 23 23

The re-read covers 23 rows, not 20. The three extra rows are cel-select-option-visible, cel-inline-grid-cell and cel-field-group-section — named as the split-out rows by the ledger's own header, in the paragraph #15802 added: "The rows split out by that change are cel-select-option-visible, cel-inline-grid-cell and cel-field-group-section." Their fail-policies corroborate it arithmetically: two fail-soft-log and one fail-closed, which is exactly the 7→9 and 7→8 delta above.

⚠️ That attribution is read from the ledger's prose, ⛔ not from git log: this checkout is shallow (git rev-parse --is-shallow-repositorytrue), so git log -S on those row ids returns the oldest commit inside the shallow window rather than the introducing one, and would have named an unrelated PR.

The 23 rows carry 46 covers keys between them.

A second stale number, same cause: the ruling names five state: 'experimental' rows. There are six today. The sixth is cel-inline-grid-cell, and its verdict is keep — see the table below. The ruling's enumeration of five survives the re-derivation; only the arithmetic around it was stale.

The five rows re-stated

row was now why the old value was a borrowing
cron-knowledge-refresh compile-error unevaluated cell says nothing evaluates the result
cron-declared-unwired (7 covers keys) compile-error unevaluated cell says NO EVALUATOR FOUND
template-prompt (2 covers keys) compile-error unevaluated cell says NO EVALUATOR FOUND
template-title-format compile-error unevaluated cell says PARSE ONLY in this repo; the one named reader is a build-time lint warning that never looks at the template text
cel-advanced-policy (2 covers keys) fail-closed unevaluated cell says (no runtime consumer yet) — a runtime refusal claimed on a slot with no runtime consumer, on a security-flavoured row

compile-error claimed the Zod parse was the refusal. That is true and it is empty: the parse judges the value's shape and never its grammar, and every row in this ledger has it. fail-closed on cel-advanced-policy is the one that cost something past legibility, which is why B was refused.

⚠️ compile-error is now an unpopulated member. All four rows carrying it were the borrowers. Left in place deliberately — see the notes at the end.

Every existing row's re-read verdict — all 23

# row state failPolicy verdict
1 rls-using enforced fail-closed keep — compileCelToFilter, AND-injected; proof cited
2 rls-check enforced fail-closed keep — compileCelToFilter + matchesFilterCondition; proof cited
3 sharing-condition enforced fail-closed keep — celToFilter at bootstrap; proof cited
4 cel-validation enforced fail-soft-log keep — celEngine via the validation runner
5 cel-hook enforced fail-soft-log keep — celEngine via the hook runner
6 cel-formula enforced fail-soft-log keep — celEngine
7 cel-field-rule enforced fail-soft-log keep — celEngine, console + server rule-validator
8 cel-select-option-visible enforced fail-soft-log keep — the cell states the fail-OPEN path explicitly ("logged with the reason and allowed through")
9 cel-inline-grid-cell experimental fail-soft-log keep — the boundary case. ⛔ NOT unevaluated. The cell names an evaluator it could not reach (the objectui renderer, outside this checkout) and reports a measured write path on which nothing refuses. unevaluated asserts an absence; this checkout cannot establish one, and asserting it would be the invented cell this ledger exists to prevent, in the other direction
10 cel-ui enforced fail-soft-log keep — SchemaRenderer + server celEngine
11 settings-visibility enforced fail-closed keepevaluateVisibility refuses the save (HTTP 400); a real runtime refusal
12 cel-action-param-option-visible enforced fail-soft-log keep — celEngine per option; the cell already scopes the claim to UI gating
13 cel-bulk-action-visible enforced fail-closed keep — faults hide the record (fallback:false) rather than acting on it
14 cel-field-group-section enforced fail-closed keep — FALSE or a faulting predicate hides the whole group; that is what separates it from cel-ui
15 cel-row-crud-visible enforced fail-closed keep — FALSE/fault hides the row button
16 cel-row-crud-disabled enforced fail-soft-log keep — a fault leaves the button enabled
17 cel-flow enforced throw keep — celEngine via the automation runtime
18 cron-job-schedule enforced throw keeptoBoundaryJobSchedule throws, contained at the call site and logged
19 cron-knowledge-refresh experimental compile-errorunevaluated re-stated
20 cron-declared-unwired experimental compile-errorunevaluated re-stated
21 template-prompt experimental compile-errorunevaluated re-stated
22 template-title-format experimental compile-errorunevaluated re-stated
23 cel-advanced-policy experimental fail-closedunevaluated re-stated — the row the ruling turns on

Two axes were re-read alongside failPolicy and deliberately left alone. mode: the five re-stated rows keep interpret. ADR-0058 D6 makes mode a property of what the surface is, not of whether anything runs it, and re-opening it is a different vocabulary question this ruling did not put. state: unchanged on every row; unevaluated is a fail-policy, not a lifecycle claim.

The load-bearing half — the new word cannot be borrowed as loosely

The ruling asks that a row on an experimental slot carrying unevaluated and a non-empty runtime enforcement be rejected by the test.

⚠️ "non-empty runtime enforcement" cannot be spelled enforcement !== ''. ExprSurface makes the cell required, so every row has a non-empty one — the five honest unevaluated rows included, whose cells are long prose. A literal emptiness check would reject exactly the rows the ruling is minting the word for.

The checkable question is what the cell says. The pin asserts three things of every unevaluated row:

  1. it states the absence it claims — the cell must carry one of NO EVALUATOR FOUND / PARSE ONLY / no runtime consumer, so the claim is reviewable rather than inferred from silence;
  2. it does not name a runtime evaluatorNAMES_RUNTIME_EVALUATOR is a detector built from the evaluator sites this ledger's own cells already name (celEngine, compileCelToFilter, ExpressionEngine.evaluate, evaluateVisibility, croner, …). Saying the words in (1) is not enough if the same cell names the thing that evaluates it;
  3. it is not state: 'enforced' — restricting the pin to experimental rows would leave enforced as the escape hatch. unevaluated means nothing reads the slot; enforced means the platform enforces it. The contradiction is refused directly.

Relationship to ADR-0058 D5. D5's matrix is keyed by (when) × (security-relevance) and every tier in it describes what happens at a site that evaluates something; it has no row for a slot with no site. unevaluated is therefore a ledger-local extension, and the FailPolicy docblock says so in those words ("ADR-0058 D5 fail-policy tiers, plus the one state D5 has no tier for"). ⛔ No ADR was edited — docs/adr/** is a governed surface, and the ruling scoped this change to the test file.

The pin carries its own anti-vacuity control, in this file's existing idiom (expect(declarations.length).toBeGreaterThan(0) on the sibling pin): an emptied or mistyped NAMES_RUNTIME_EVALUATOR would make assertion (2) pass over everything. The control asserts the detector fires on every mode: 'compile' row — the rows another pin in the same file already requires to name the canonical compiler — so a broken detector reds instead of passing silently.

⚠️ Stated plainly, because a pin that overclaims is the defect this card is about: no regex reads prose for honesty. What (1) and (2) mechanically refuse is the shape the four older members were borrowed in — a cell that describes a site and leaves the reader to infer what happens to a bad expression. The truth of the sentence remains a review question; the pin makes there be a sentence to review.

Demonstrated RED — four legs, each on a distinct assertion

Implementation committed first (6dc219bebe), then ablated; the legs below were re-run against that exact head. Every leg proves the mutation reached disk (blob hash differs from the HEAD blob, injected/removed anchor counts) before the run is believed, restores under an EXIT/INT/TERM trap using git checkout HEAD -- (never the bare form, which restores from a polluted index), and proves byte-identity against the HEAD blob afterwards. Exit codes captured before any pipe.

HEAD blobs at that commit: ledger 2dbac23c974541fc897a16739770789bda1cf5ff, test 02493cbe009bc4c7660493b00632404b782f31e9.

leg mutation on-disk proof result
0 baseline none GREEN — 5 passed
A the forbidden combination cel-advanced-policy.enforcement'@objectstack/formula celEngine (interpret) via the advanced-policy runner' (experimental + unevaluated + a cell naming a runtime evaluator) anchor 0, injected 1, blob c7aa4cd5 ≠ HEAD RED — 1 failed / 4 passed, on assertion (1)
A2 words are not enough same cell → '(no runtime consumer yet) — evaluated by @objectstack/formula celEngine' (keeps the disclaimer, names the evaluator) anchor 0, injected 1, blob daae2753 ≠ HEAD RED — on assertion (2), isolated
B the escape hatch cel-advanced-policystate: 'enforced', still unevaluated anchor 0, injected 1, blob b3ed4f5d ≠ HEAD RED — on assertion (3)
C anti-vacuity control NAMES_RUNTIME_EVALUATOR → a regex matching nothing anchor 0, injected 1, blob 088cdb5c ≠ HEAD RED — the control fires on rls-using, so assertion (2) is not vacuous
D restored none both blobs byte-identical to HEAD, git diff HEAD empty GREEN — 5 passed

Leg A's failure text, verbatim:

AssertionError: cel-advanced-policy: failPolicy 'unevaluated' but the enforcement cell
never states the absence it claims. Say it in the cell — 'NO EVALUATOR FOUND',
'PARSE ONLY', or 'no runtime consumer' — so the claim is reviewable rather than
inferred from silence: expected false to be true

Verification

Heavy runs went through scripts/pm/os-verify-lock.sh; verdicts are quoted from the lock's own VERDICT line, never a bare exit code.

  • Suitepnpm --filter @objectstack/dogfood exec vitest run test/expression-conformance.test.tsVERDICT command-exit 0, 1 file / 5 tests passed (4 before this PR; the new pin is the fifth).
  • Typecheck — dependency closure built first (pnpm --workspace-concurrency=2 --filter '@objectstack/dogfood^...' build), then tsc --noEmit --listFilesVERDICT command-exit 0, 0 error TS, and --listFiles names both edited files in the program, so the verdict really covers them.
  • Affected packagesturbo ls --affected against this branch's merge base: @objectstack/dogfood only.
  • Control charactersgrep -naP over both files finds none, beyond pnpm check:nul-bytes.

Gate union

node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands derived 48 runnable commands from this diff's two paths; all 48 were run against the final head 6dc219bebe, and the ran-list was reconciled with --ran. Results are quoted from each gate's own output, with the exit code captured before any pipe.

48 derived · 48 run · 0 NOT-MEASURED · 0 UNRUN, and every one exited 0. dispatch-gates --ran confirms it:

Run reconciliation — 48 derived, 48 run, 0 NOT-MEASURED, 0 UNRUN.
✓ dispatch-gates --ran: 48 derived famil(ies) accounted for — 48 run, 0 NOT-MEASURED.

⚠️ Two gates were NOT MEASURED on a first pass and are reported as measured only because they were re-run to a real verdict — an exit 3 is a prerequisite failure, never a pass:

  • pnpm check:dual-build-cjs-loads — first run exit 3, PREREQUISITE NOT MET ... this gate reads built output, naming 8 packages with no dist/. Re-run after the dependency-closure build: exit 0.
  • pnpm check:type-check-debt — first run exit 3 after 744s, a V8 FATAL ERROR: Ineffective mark-compacts near heap limit under the standard --max-old-space-size=4096. Re-run at 6144 (raised deliberately, for that measured OOM, on a box with 14 GB available): exit 0 in 130s. The gate's own text is explicit that its exit 3 "is NOT a pass and NOT a finding".

Slowest four, for whoever schedules this next: pnpm check:query-options-erasure (259s) · pnpm check:type-check-debt (130s) · pnpm check:slot-lookup (117s) · node scripts/check-comment-mask-corpus.mjs (110s).

⚠️ Staleness, named rather than hidden. dispatch-gates reports this tree as behind origin/main in files the derivation reads (.github/workflows/lint.yml, package.json). Diffed: main has added exactly two families since this branch point, and neither reaches this diff. pnpm check:pm-widening-tells is a --self-test-only step whose own comment records that its input is "a DIFF supplied by its caller, never a file in the tree" — checker-health, not a verdict on any diff. pnpm check:scaffold-emission-policy declares POLICY_SOURCE = 'packages/cli/src/commands/init.ts' and two docs .mdx pages as its inputs; this PR touches none of them. The remaining always-runs tail is CI's to answer.

⚠️ Declared narrowing: the whole @objectstack/dogfood suite was not run locally. It boots real example apps ~130 times and CI shards it three ways; this diff touches one test file and its ledger and no runtime source. The suite's own file was run in full. The rest is declared to CI.

Changeset

None — skip-changeset. @objectstack/dogfood is "private": true; this diff is confined to it and releases nothing from any package. That is the workflow's own stated criterion for the label ("such a PR releases nothing"). The label is applied on this PR at open time, not left for the Check Changeset job to red first.

验收备注 — found, deliberately not filed

  1. compile-error is now an unpopulated member of FAIL_POLICIES. All four rows that carried it were borrowers, so it now names no row. Retiring it is a live question — [finding] CronExpressionInputSchema / TemplateExpressionInputSchema fix the dialect only on the bare-string arm — the envelope arm accepts any declared dialect, so a cron-typed slot parses { dialect: 'cel', source } green #15028 records that it reads stronger than it is — but an unused vocabulary member is dead code, which is not a filing category, and retiring it is a decision this ruling did not make. Noted, not filed.

  2. The reverse pin was considered and deliberately NOT written. The symmetric rule — a row whose cell declares no evaluator and names none must be unevaluated — is green on today's ledger (the three disclaimer phrases match exactly the five unevaluated rows and no other row, measured). It is the leg that would have caught cel-advanced-policy borrowing fail-closed in the first place. It is not here on purpose: its population is defined by matching three phrases of prose, so a future unwired row that spells its absence any other way passes it silently while the pin reads as "the old words can no longer be borrowed". A partial gate that reads as complete is the exact defect family this ledger catalogues, and adding one to close this card would be a poor trade. The forward direction has no such problem — it governs every row carrying a machine-readable field value. Noted, not filed.

  3. cel-inline-grid-cell carries a runtime fail-policy over a surface its own cell says was NOT MEASURED IN THIS REPO. Row 9 above. Its fail-soft-log is defensible on the measured half (the write path reads nothing, so nothing refuses), and its note already sets the condition for re-stating it. But the ledger has no word for "an evaluator exists and is out of reach", which is a third dialect of the same legibility question this card answers for "no evaluator at all". Adjacent, not this card, and not a defect against a declared contract. Noted, not filed.

  4. A pre-existing editing artifact in the ledger header, untouched. Lines 31-32 of expression-conformance.ledger.ts read "Two limits survive and / One limit survives and is worth knowing…" — a half-replaced sentence left by an earlier edit, plainly visible in the diff's context lines. It is a documentation nit in a file this PR edits, not a defect in anything the ledger asserts, and this PR's diff is deliberately confined to the vocabulary change. Noted, not filed.


Generated by Claude Code

…tes this"

`FAIL_POLICIES` had four members and all four describe what an EVALUATOR does
when the expression is bad. Five rows classify slots that have no evaluator at
all, so each had to borrow a member claiming something stronger: four carried
`compile-error` (the Zod parse is the refusal — true, and a property every row
in the ledger shares) and `cel-advanced-policy` carried `fail-closed`, a RUNTIME
refusal, on a slot whose own `enforcement` cell reads `(no runtime consumer
yet)`. On a security-flavoured row that reads as a security guarantee.

Adds `unevaluated` and re-states all five rows with it. Every other row's
`failPolicy` was re-read against the new member and left where it was.

Minting a word that could be borrowed as loosely would reproduce the defect one
member wider, so the member arrives with a pin. "Non-empty runtime enforcement"
cannot be checked as `enforcement !== ''` — `ExprSurface` makes the cell
required, so every row has one. The checkable question is what the cell SAYS:
an `unevaluated` row must state the absence it claims and must not name a
runtime evaluator site, and it cannot be `state: 'enforced'`. The detector
carries a positive control against the COMPILE rows another pin already
requires to name the canonical compiler, so an emptied regex reds instead of
passing vacuously.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YFY46JydE1gMxQG1TqBcMZ
@claude claude Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 7, 2026
@github-actions github-actions Bot added the size/m label Sep 7, 2026
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 813d6c5b3ddbb35ff282cbc338574cbfe98c9093packageMentionDocs.

@github-actions github-actions Bot added the tests label Sep 7, 2026
@os-sales
os-sales marked this pull request as ready for review September 7, 2026 16:59
@os-sales
os-sales enabled auto-merge September 7, 2026 16:59
@os-sales
os-sales added this pull request to the merge queue Sep 7, 2026
Merged via the queue into main with commit 2539f4a Sep 7, 2026
40 checks passed
@os-sales
os-sales deleted the claude/issue-15533-failpolicies-unevaluated branch September 7, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

2 participants